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ABSTRACT 

The  current  medical  equipment  maintenance  model  is  “on  site  and  physical",  rather  than  “remote  and 
virtual".  There  is  no  comprehensive  or  standardized  remote  diagnostic  access  (RDA)  capability  for 
biomedical  technicians  to  troubleshoot  and  resolve  problems  remotely  and  securely.  Additionally,  the  current 
operations  model  is  “reactive”,  rather  than  “proactive” ,  in  terms  of  early  problem  detection  and  prevention. 
The  lack  of  visibility  to  the  health  of  medical  equipment  and  the  need  to  be  on  site  for  problem  resolution, 
coupled  with  frequent  rotations  and  scarcity  of  medical  equipment  technicians,  continue  to  cause 
considerable  downtime  of  critical  medical  equipment  densities  and  is  detrimental  to  the  health  care  support  to 
our  warfighters. 

Complex  medical  equipment  such  as  Magnetic  Resonance  Imaging  (MRI),  Computed  Tomography  (CT), 
Computed  Radiology  (CR)  scanners,  along  with  ultrasonic  and  laboratory  devices  are  critical  to  the  patients’ 
treatment  regimen.  Unscheduled  delays  and/or  extensive  downtime  of  the  equipment  severely  hamper  the 
physicians’  ability  to  diagnose  and  treat  a  patient’s  injury  or  medical  condition.  Complexity  often  requires 
the  local  maintainer  to  rely  on  external  support  or  SMEs,  many  times  from  the  equipment’s  manufacturer,  to 
assist  in  the  diagnosis  and  repair.  This  “wait  and  see"  method  coupled  with  the  lack  of  any  pre-screening 
capability  to  identify  troubled  areas,  worn  parts,  or  signal  “out  of  tolerance  ”  modalities  can  cause  additional 
delays. 

While  medical  equipment  manufacturers  offer  variations  of  managed-setv’ices  to  monitor  and  maintain 
medical  equipment  in  service  for  commercial  installations,  these  offerings  often  can  only  support  individual 
manufacturers  ’  own  brand  and  model  of  medical  equipment.  Furthermore,  their  service  architecture  and  the 
providers’  infrastructures  generally  do  not  comply  with  the  government’s  security  protocols  and  regulations. 
As  a  result,  virtually  all  the  offerings  do  not  have  the  authority  to  operate  (ATO)  within  the  government 
networks  due  to  the  lack  of  accreditations  and  compliance.  Compounding  the  problem  is  the  fact  that 
manufacturers  are  reluctant  to  release  proprietary  hardware  and  software  specifications,  access  protocols, 
application  programming  interfaces  (API),  or  software  development  kit  (SDK)  to  allow  independent 
development,  integration,  and  support  for  telemaintenance  in  medical  equipment  operations. 

The  subject  of  this  paper  is  to  summarize  the  research  and  development  of  a  new  RDA  capability,  and  a 
proactive  monitoring  system  as  the  foundation  of  the  new  medical  equipment  telemaintenance  framework. 
The  RDA  helps  transition  the  maintenance  model  from  “ physical ”  to  “remote”,  while  the  monitoring  system 
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helps  transition  the  operations  model  from  “reactive”  to  “proactive”.  In  essence,  the  monitoring  system  is 
enabling  the  condition-based  maintenance  ( CBM+ )  practice  by  proactively  seeking  and  detecting  conditions 
on  medical  equipment  that  may  trigger  a  maintenance  work  order  on-demand.  As  such,  it  will  also  help 
transition  the  management  model  from  a  “ time-based ”  to  “condition-based”  practice.  The  research  outcome 
is  a  unified  remote  diagnostic  access  gateway,  the  Teleconsole,  with  an  efficient  methodology  for  provisioning 
secure  remote  access  for  medical  equipment  diagnostics.  This  unique  RDA  gateway  provides  all  the 
necessary  tools  to  enable  proprietary  and  legacy  medical  equipment,  including  those  without  native  RDA 
support,  to  become  telemaintenance-ready.  The  new  methodology  provides  technicians  with  the  ability  to 
remotely  perform  diagnostic  tasks  and  resolve  problems  without  any  time  and  physical  constraints.  The 
research  continues  with  the  development  of  a  methodology,  based  on  the  concept  of  computer  vision  (neural 
networks)  working  in  conjunction  with  proven  techniques  such  as  macro  automation,  to  train  computer 
systems  to  perform  monitoring  tasks  on  behalf  of  the  technician  around  the  clock.  This  research  is  in 
progress. 

1.0  INTRODUCTION 

For  many  decades,  telemaintenance  and  condition-based  maintenance  have  been  well  established  ‘best 
practices’  for  the  Information  Technology  (IT)  operations.  However,  most  recent  independent  surveys  still 
suggest  that  these  capabilities  are  virtually  non-existent,  or  not  an  enterprise-wide  capability,  in  equipment 
operations  outside  the  realm  of  IT  services,  such  as  healthcare,  utilities,  communications,  and  other  sectors. 

Telemaintenance  is  accomplished  by  a  remote  diagnostic  access  (RDA)  capability  that  enables  a  remote 
technician  to  perform  diagnostic  tasks  on  equipment  without  having  to  be  onsite,  or  the  ability  for  the  local 
maintainer  to  collaborate  with  remote  subject  matter  experts  (SMEs)  assisting  in  the  troubleshooting  and 
repair  of  the  equipment. 

Condition-based  maintenance  (CBM)  is  accomplished  by  real-time  sensors  deployed  to  monitor  the  health  of 
the  equipment.  When  an  impending  problem  is  detected,  the  condition  serves  as  a  trigger  for  the  responsible 
technician  to  perform  a  specific  maintenance  task  on  the  equipment. 

The  research  and  development  initiative  is  to  establish  a  new  medical  equipment  telemaintenance  framework 
consists  of  two  key  objectives:  the  RDA  capability  to  enable  secure  remote  access  for  equipment  diagnostics, 
and  the  Proactive  Monitoring  capability  to  enable  early  problem  detection  by  leveraging  the  new  RDA 
capability.  Considering  the  RDA  capability  as  “paving  the  roads”  on  which  diagnostic  activities  and 
equipment  statistical  information  can  be  transported,  then  the  Proactive  Monitoring  capability  can  be  viewed 
as  “problem  sensors”  for  early  problem  detections.  Being  able  to  detect  impending  equipment  problems 
before  they  occur  is  the  beginning  of  a  transformation  from  the  current  time-based  maintenance  methodology 
to  a  condition-based  maintenance  (CBM+)  operations  model.  All  of  these  are  fundamental  building  blocks  of 
the  telemaintenance  platform;  furthermore,  they  align  with  the  requirements  and  objectives  of  the  Hospital  of 
the  Future  (HOF)  initiatives  as  well. 


2.0  RDA  AND  CBM  IMPLEMENTATION  CHALLENGES 

In  the  IT  world,  console  access  has  been  standardized  with  the  RS-232  protocol.  Whether  a  DB25  (outdated), 
DB9  or  RJ45  console  port  is  on  the  device,  the  same  software  and  methodology  is  used  to  access  network 
devices  or  servers. 
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2.1  Proprietary  Software  and  Communications  Protocols  (Standardization) 

It  isn’t  the  same  for  medical  equipment.  Each  piece  of  equipment  may  have  a  different  communication 
interface,  as  well  as  different  software  required  to  perform  the  maintenance.  Original  equipment 
manufacturers  (OEMs)  use  proprietary  software  for  the  console  access,  placing  dependencies  on  the 
manufacturers  to  provide  management  and  diagnosis  capabilities.  This  is  good  for  their  bottom-line  as  they 
provide  managed  services  offerings,  often  times  for  different  makes  and  models  within  their  own  product 
lines. 

2.2  Non-disclosure  by  OEM  (Interoperability) 

Adding  to  this  vendor  “lock-in”  is  the  fact  that  they  are  reluctant  to  release  proprietary  hardware  and  software 
specifications,  access  protocols,  application  programming  interfaces  (API),  or  software  development  kits 
(SDK)  to  allow  independent  development,  integration,  and  support  for  telemaintenance.  Without  the  ability  to 
integrate  or  consolidate  these  individual  and  diverse  management  points,  from  various  OEMs,  the  enterprise 
does  not  have  a  standardized  and  centralized  management  platform  on  which  complex  data-mining  can  be 
performed  and  analyzed. 

2.3  IT  and  Biomed  Operational  Gaps  (Alignment/Convergence) 

Some  of  the  more  complex  medical  equipment  are  partially  available  via  remote  access,  due  to  the  fact  that 
they  have  two  management  points.  The  first  is  a  ‘computer’  part  that  runs  a  specialized  OS,  such  as  Microsoft 
XP  Embedded,  along  with  custom  software  used  by  the  technician  to  operate  the  equipment.  The  second  is  a 
‘mechanical’  part  that  is  controlled  by  circuit  boards  and  is  accessible  by  the  technician  via  the  serial  console 
and  running  proprietary  diagnostic  software.  To  implement  full  RDA  for  medical  equipment,  the  IT 
organization  assisting  the  efforts  often  does  not  understand  the  dual  management  points.  If  the  IT  engineers 
recommend  some  in-band  access  solutions,  such  as  virtual  network  computing  (VNC),  remote  desktop 
protocol  (RDP),  or  virtual  private  network  (VPN),  then  the  solution  negates  the  management  point  on  the 
medical  equipment  that  can  only  be  accessed  via  out-of-band  console.  Conversely,  if  the  suggested  solution  is 
serial  console  access,  then  the  equipment  cannot  be  accessed  via  in-band  services  as  mentioned  above,  or 
other  out-of-band  console  such  as  those  using  USB  and  KVM  interfaces.  As  such,  IT  organizations’ 
recommended  solutions,  based  on  products  designed  for  IT  operations,  generally  do  not  meet  the  requirements 
of  biomed  telemaintenance,  such  as  form  factor,  in-band  versus  out-of-band  access,  serial  console  versus  USB 
console,  and  text -based  console  versus  graphical  console  (KVM  products). 

2.4  The  Lack  of  CBM  Capability 

Monitoring  of  IT  components  is  well  established  and  an  operational  best  practice  to  provide  visibility  to  the 
health  of  IT  resources.  It  also  provides  the  predictability  of  when  a  problem  will  occur  so  that  the 
administrators  can  proactively  diagnose  and  resolve  the  problem  before  they  can  impact  the  overall 
operations.  Proactive  monitoring  in  IT  uses  standard  protocols  and  tools  adhering  to  those  protocols  over  the 
‘in-band’  network.  Unfortunately,  that  same  methodology  is  not  applicable  to  medical  technology  operations. 
First,  OEMs  don’t  adhere  to  standards  and  continue  to  use  proprietary  management  tools  to  maintain  vendor 
lock-in.  Second,  device  access  for  management  and  maintenance  is  ‘out-of-band’  through  the  console  port 
and  typical  IT  monitoring  tools  are  not  capable  of  utilizing  that  access  method.  With  a  properly  implemented 
RDA  capability  in  support  of  telemaintenance,  it  can  also  serve  as  the  necessary  transport  for  the 
implementation  of  the  CBM  capability. 
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All  of  the  factors  above  contribute  to  the  challenges  organizations  face  to  maintain  a  fully  functioning  and 
efficient  health  care  facility.  Some  of  the  shortcomings  of  this  on-site  approach  include: 

1)  It  is  slow.  Because  the  technician  or  manufacturer  support  personnel  need  to  go  on-site  for 
maintenance,  the  travel  time  needs  to  be  accounted  for.  In  addition,  personal  schedules  can  impact 
time  to  repair  as  organizations  are  reliant  on  a  subset  of  individuals  to  perform  the  maintenance. 

2)  It  is  costly.  Travel  costs  are  quickly  becoming  a  large  expense  in  organizations.  Because  all  work 
needs  to  be  on-site,  those  travel  expenses  can  quickly  grow  to  be  a  financial  constraint.  But,  it  is  also 
costly  in  another  form  -  that  of  human  health  and  lives.  People  rely  on  this  equipment  on  a  daily 
basis  for  emergency  medical  service  and  chronic  care  management,  which  can  cost  people  their 
health,  or  even  lives,  if  the  equipment  is  down  or  not  functioning  properly. 

3)  It  is  reactionary.  Due  to  the  on-site  nature  of  maintenance,  it  typically  ends  up  being  done  in  a 
break-fix  manner.  This  contributes  both  to  the  cost  and  time  necessary  to  make  the  maintenance 
repairs.  Patient  safety  is  directly  affected  by  this.  It  has  been  recognized  by  the  World  Health 
Organization  that  1  in  10  patients  around  the  world  are  impacted  by  health  care  errors.  It  is  unknown 
how  many  of  those  errors  are  as  a  result  of  out-of-maintenance  or  malfunctioning  medical  equipment. 

3.0  PREVIOUS  RDA  IMPLEMENTATION  ATTEMPTS 

An  earlier  attempt  was  made  to  implement  RDA  capability  for  medical  equipment  using  teleconferencing 
tools  that  worked  effectively  for  telemaintenance  of  aircrafts  and  vehicles.  This  type  of  system  is  generally 
made  up  of  a  video  camera,  an  audio  device  (microphone,  speaker,  or  handset),  a  computer  with  pre-installed 
diagnostic  software,  a  communication  device  for  network  connectivity  (wired,  WIFI,  satellite  link,  etc.),  and 
an  optional  battery  for  mobility.  The  system  utilizes  video  for  remote  over-the-shoulder  viewing  during 
troubleshooting  hardware  failures,  and  the  audio  gears  allow  the  local  maintainer  to  communicate  with  the 
remote  SMEs.  When  the  system  was  used  in  medical  equipment  telemaintenance,  the  RDA  capability 
provided  by  the  system  was  found  to  be  inadequate.  Some  of  the  limitations  are: 

1)  In  general,  medical  equipment  troubleshooting  requires  proprietary  diagnostic  software  supplied  by 
the  equipment  manufacturer  and  they  must  run  on  the  local  maintainer’s  laptop.  The  software  also 
requires  a  physical  serial  cable  connecting  from  the  laptop  to  the  medical  equipment’s  console  port  (a 
DB9  port  is  common  although  newer  equipment  comes  with  a  USB  port  instead).  Since  the  software 
is  very  specific  to  the  brand,  model,  and  firmware  versions  of  the  medical  equipment  being  diagnosed, 
the  local  maintainer  must  use  the  software  that  matches  the  hardware  component  to  ensure 
compatibility.  This  means,  if  the  teleconference  system  is  used  for  medical  equipment 
telemaintenance,  the  computer  found  in  the  system  must  have  all  the  diagnostic  software  pre-installed, 
including  different  version  of  the  same  software.  Even  if  it  is  possible  to  pre-load  all  the  diagnostic 
software  from  all  the  manufacturers  into  all  the  teleconference  systems’  computer,  it  is  impossible  to 
keep  the  versions  of  all  software  up-to-date  on  these  systems. 

2)  The  teleconference  system  is  usually  equipped  with  one  serial  port  to  support  wired  console  access. 
This  means  the  local  maintainer  cannot  diagnose  several  equipments  simultaneously,  or  he  cannot 
efficiently  troubleshoot  on  complex  equipment  with  multiple  console  ports,  such  as  the  MRI. 

3)  The  teleconference  system  is  designed  for  on-demand  applications.  That  is,  a  local  maintainer  would 
connect  this  device  to  the  medical  equipment  at  the  time  of  need  -  when  the  equipment  needs  repair. 
This  method  is  not  a  long-term  solution  given  one  of  the  medical  telemaintenance’s  objectives  is  the 
proactive  monitoring  capability,  which  would  require  the  teleconference  system  to  have  a  persistent 
connection  to  all  the  medical  equipment  being  monitored. 
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Another  previous  attempt  to  establish  RDA  for  medical  telemaintenance  was  the  use  of  conventional  remote 
access  solution  designed  for  telecommuters  or  teleworkers  in  the  Information  Technology  (IT)  world,  such  as 
a  virtual  private  network  (VPN)  and  other  secure  access  gateways.  This  type  of  solution  enables  a  remote  user 
to  connect  to  a  computer  at  the  office,  or  allows  an  IT  support  personnel  to  remotely  access  the  user’s 
computer  to  troubleshoot  a  software  issue.  When  the  solution  was  tested  for  medical  equipment 
telemaintenance,  it  was  also  proven  inadequate.  The  limitations  are: 

1)  Telecommuter  systems  (hardware,  software,  or  both)  are  in-band  or  network-based  access  solution. 
They  work  effectively  when  the  remote  computer  and  its  operating  system  are  healthy.  This  type  of 
solutions  generally  does  not  provision  access  for  out-of-band  methods,  such  as  serial  console, 
universal  serial  bus  (USB)  console,  or  keyboard  video  mouse  (KVM)  consoles.  For  medical 
equipment  diagnosis  or  calibrations,  out-of-band  access  method  is  required.  As  such,  out-of-band 
access  devices  must  be  added  to  the  in-band  access  solution.  The  resulting  RDA  solution  becomes 
very  bulky  and  difficult  to  manage,  as  explained  below. 

2)  Conventional  IT -centric  access  solutions  are  generally  not  designed  for  medical  operations 
environment,  such  as  the  ER,  mobile  hospitals,  and  etc.  First,  the  hardware  form  factor  is  designed 
for  mounting  on  a  rack.  For  medical  equipment  telemaintenance,  the  form  factor  should  be 
miniaturized  so  that  the  access  device  can  be  embedded  inside  and  become  an  integral  part  of  the 
medical  equipment. 

3)  Commercial  off  the  shelf  (COTS)  products  for  out-of-band  access  are  available,  such  as  those  serial- 
over-FAN  devices  designed  for  Supervisory  Control  And  Data  Acquisition  (SCADA)  applications, 
and  KVM-over-FAN  for  headless  server  applications.  This  method  of  bundling  different  brands  of 
out-of-band  devices,  with  each  one  being  a  stand-alone  or  point  solution,  to  create  the  RDA 
capability  for  medical  equipment  telemaintenance  is  not  ideal  because  it  is  impossible  to  integrate 
all  these  COTS  products  into  a  single  management  point. 


4.0  RDA  RESEARCH  AND  DEVELOPMENT 

In  order  to  understand  the  issues  pertaining  to  implementing  RDA  for  medical  equipment  telemaintenance,  the 
next  few  sections  summarize  various  subjects  that  are  relevant  to  the  research  approach  and  prototype  strategy 
to  be  discussed  later  in  the  report: 

4.1  In-band  and  Out-of-band  RDA  for  Medical  Equipment 

In-band  access  is  a  “network-based”  access  method,  which  allows  a  technician  to  access  the  medical 
equipment  via  the  network.  This  implies  that  the  Operating  System  (OS)  and  the  access  application  servers 
running  on  the  OS  in  the  medical  equipment,  such  as  the  Virtual  Network  Console  (VNC)  server,  File 
Transfer  Protocol  (FTP)  server,  Secure  Shell  (SSH)  server,  etc.,  must  be  fully  functional. 

Whenever  the  medical  equipment  is  having  a  problem  at  the  OS  or  firmware  level  (hardware),  the  network 
services  supporting  remote  access  often  become  unavailable;  and  consequently,  the  equipment  can  not  be 
reached  via  the  network.  This  is  because  the  network  driver  and  the  associated  services  are  not  running  when 
the  OS  is  not  operational. 

Out-of-Band  is  a  “non-network”  access  method,  which  allows  the  technician  to  access  the  medical  equipment 
using  a  physical  connection  between  the  user  and  the  console  found  on  the  equipment.  The  console  port  can 
be  one  of  three  types:  serial  (DB9,  DB25),  USB,  and  KVM.  Universally,  the  purpose  of  the  console  port  is  to 
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allow  administrative  access  for  configuration,  troubleshooting,  or  calibration  when  the  equipment  is  either  not 
accessible  via  its  network  interface;  or  in  some  cases,  certain  administrative  tasks  can  only  be  performed  via 
the  console.  The  term  “console  access”  is  analogous  to  an  out-of-band  access. 

4.2  Understanding  Console  Access  for  IT  Operations 

In  IT  operations,  network  and  server  administrators  typically  use  in-band  access  to  perform  routine 
maintenance  tasks  while  the  device  is  fully  functional.  However,  for  configuration  changes  or  when  the 
device  is  non-functional  due  to  component  failures,  they  must  rely  on  the  out-of-band  or  console  access  to 
configure  and  troubleshoot  the  problem. 

A  network  technician  generally  uses  a  serial  console  access  method  for  their  remote  diagnostic  access.  One 
thing  to  note  is  that  the  serial  console  access  in  the  IT  world  has  been  standardized  such  that  the  software  used 
to  connect  to  a  serial  console  on  one  brand  of  network  router  is  the  same  for  accessing  the  console  on  another 
brand;  furthermore,  the  same  software  can  be  used  to  connect  to  the  serial  console  on  different  brands  of 
server.  Specifically,  the  common  protocols  are  Telnet  and  SSH  for  “terminal  emulation  access”,  the  actual 
implementations  are  known  as  reversed -Telnet  and  reversed-SSH,  respectively. 

Another  standard  in  IT  operations  for  serial  console  access  is  the  use  of  a  serial  console  server,  a  device  that 
accepts  the  user’s  commands  via  the  network  protocols  and  then  relays  the  messages  to  the  console  on  the 
device  via  a  serial  communication  protocol.  With  this  device,  along  with  the  standardized  access  protocols, 
remote  diagnostic  access  capability  for  IT  operations,  even  for  out-of-band  access,  have  been  achieved  and  is 
a  common  practice. 

A  new  type  of  serial  console,  based  on  the  USB  standard,  appeared  in  recent  years.  The  USB  console  access 
method  is  completely  different  from  the  serial  console  and  it  is  still  uncommon  even  for  IT  practices.  The 
remote  diagnostic  access  for  IT  equipment,  based  on  the  USB  console  access  method,  is  almost  non-existent. 

A  server  administrator  generally  uses  a  network-enabled  KVM  console  access  method  to  remotely  maintain 
servers.  The  KVM  console  is  simply  a  device  that  takes  the  analog  video  output  signals  from  the  VGA  port 
on  the  server  and  converts  it  to  digital  signals  so  that  they  can  be  transmitted  over  the  network.  The  KVM 
console  allows  the  administrator  to  take  control  of  the  server  hardware  (with  the  ability  to  control  the  mouse 
and  keyboard  while  viewing  the  monitor  output)  as  if  he  /  she  is  physically  in  front  of  the  server.  The  KVM 
access  method  used  in  IT  operations  is  well  established  and  can  support  virtually  all  hardware  and  OS 
platforms. 

4.3  The  Development  of  a  RDA  Gateway 

As  part  of  the  research,  commercial  off-the-shelf  (COTS)  IT  products  were  acquired  to  implement  all  access 
methods  for  all  the  medical  equipment  in  the  lab.  At  the  conclusion  of  this  experiment,  a  preliminary  solution 
was  established  and  it  was  able  to  provide  RDA  to  all  the  equipment,  even  those  with  no  native  or  built-in 
RDA  functions.  The  solution  consists  of  a  large  collection  of  hardware  and  software  products  from  various 
vendors.  While  this  solution  can  provide  the  full  RDA  capability  for  the  medical  equipment,  it  was  deemed 
too  expensive,  complex,  and  bulky  to  be  feasible  for  medical  applications. 

It  was  decided  that  a  new  RDA  device  must  be  custom  built  to  provide  all  the  RDA  functions  without  the  high 
cost,  bulk,  and  complexity.  The  result  is  the  development  of  a  device  known  as  the  Teleconsole.  The 
Teleconsole  is  a  unique  hardware  appliance  that  has  quantified  all  possible  RDA  methods  and  integrated  them 
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into  a  single  device  -  one  with  a  comprehensive  set  of  ports,  embedded  software,  security,  and  wired  and 
wireless  capabilities  that  can  universally  transform  medical  device  with  no  built-in  remote  diagnostic 
capability  into  one  that  is  fully  telemaintenance -ready. 

Teleconsole  provides  technicians  with  the  ability  to  remotely  perform  diagnostic  tasks  and  resolve  problems 
without  any  time  and  physical  constraints.  With  such  a  new  RDA  capability,  a  local  maintainer  can 
collaborate  with  subject  matter  experts  (SMEs)  via  the  “over  the  shoulder”  view  of  the  medical  device;  or  a 
remote  technician  can  “reach  in”  via  secure  access  to  calibrate  machine  components,  retrieve  error  logs,  or 
upgrade  configuration  files  -  all  of  which  can  be  achieved  through  an  extensive  set  of  RDA  functions  that  are 
agnostic  to  the  brand,  make,  and  model  of  the  failing  medical  equipment. 

IT  support  personnel  are  accustomed  to  having  the  ability  to  access  all  equipment  remotely,  including  the 
console  ports  necessary  for  ‘out-of-band’  access.  The  latter  piece  has  been  solved  in  the  IT  world  through  the 
use  of  a  serial  console  server  -  a  device  which  accepts  communications  over  the  network  and  transfers  them  to 
the  standardized  RS-232  protocol  and  out  the  physical  console  port  connected  to  the  network  device  or  server. 
By  combining  standard  in-band  access  with  the  functionality  of  a  serial  console  server,  it  provides  alignment 
with  the  typical  access  methods  that  IT  is  familiar  with. 

Equipped  with  6  DB9  serial  ports  and  4  USB  ports  -  as  well  as  network  interfaces,  the  Teleconsole  can 
provide  in-band  and  out-of-band  access  methods  in  a  single  platform  to  both  management  points  on  even  the 
most  complex  medical  equipment.  The  in-band  allows  remote  access  to  devices  with  an  integrated  ‘computer’ 
component,  while  the  out-of-band  access  is  used  for  the  ‘mechanical’  component  of  the  medical  equipment. 

OEM  supplied  diagnostic  software  has  been  developed  to  communicate  directly  with  the  console  port  on  the 
medical  equipment.  The  Teleconsole  allows  the  software  to  think  it  is  doing  so  even  though  the  technician’s 
laptop  is  not  connected  to  any  medical  equipment.  To  accomplish  this,  a  few  steps  are  needed.  First,  a  virtual 
communications  port  needs  to  be  created  on  the  technician’s  computer  so  the  application  thinks  that  it  is 
speaking  directly  with  the  device  via  the  COM  port  and  a  physical  cable.  Next,  a  piece  of  software  running  on 
the  same  computer  needs  to  accept  the  traffic  from  the  virtual  COM  and  transmit  it  over  the  network  to  the 
serial  console  server  physically  connected  to  the  medical  equipment.  Lastly,  the  serial  console  server  transfers 
the  communication  directly  to  the  console  port  on  the  medical  equipment  using  the  raw  serial  protocol.  The 
same  process  using  a  virtual  port  and  software  transmitting  communication  over  the  network  to  the  console 
server  is  also  applicable  for  USB  access. 


5.0  PROACTIVE  MONITORING  RESEARCH  AND  DEVELOPMENT 

In  order  to  understand  the  issues  pertaining  to  the  proactive  monitoring  of  medical  equipment,  it  is  necessary 
to  understand  the  different  characteristics  of  managing  and  supporting  IT  devices  versus  medical  equipment. 

5.1  Proactive  Monitoring  of  IT  Resources 

Proactive  monitoring  of  IT  infrastructure  components,  such  as  networks,  servers,  applications,  databases,  and 
etc.  has  been  well  established.  The  monitoring  and  trending  of  IT  component  utilization,  performance,  and 
availability  are  critical  and  essential  operational  best  practices.  The  visibility  to  the  health  of  the  IT  resources 
provides  the  predictability  of  when  a  problem  will  occur  so  that  the  administrators  can  proactively  diagnose 
and  resolve  the  problem  before  they  can  impact  the  overall  operations.  It  would  be  impossible  for  any  mission 
critical  IT  infrastructure  to  exist  and  functioning  optimally  without  the  proactive  monitoring  and  problem 
detection  capability. 
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Proactive  monitoring  is  easy  to  implement  for  IT  operations  because  there  is  an  abundant  of  products,  as  well 
as  standards,  API,  and  SDK  that  allow  a  monitoring  tool  to  be  able  to  communicate  with  the  managed  devices 
for  data  collection,  statistic  gathering,  and  etc.  For  the  most  part,  hardware  and  software  vendors  in  the  IT 
world  are  very  willing  to  cooperate  by  either  adhering  to  pre-defined  management  standards;  or  they  release 
proprietary  specifications  so  their  hardware  and  software  can  be  managed  by  3ld-party  tools.  If  a  vendor  is 
unwilling  to  release  such  information,  then  enterprise  customers  will  simply  refuse  to  use  their  products. 

5.2  Proactive  Monitoring  of  Medical  Equipment 

Medical  technology  operations  currently  do  not  have  a  well  defined  management  strategy  across  the  board.  It 
is  still  relatively  primitive.  The  main  reason  is  that  medical  technology  operations  are,  for  the  most  part, 
autonomous.  And  each  silo  of  medical  technology  operations  places  heavy  dependency  on  the  equipment’s 
manufacturer  to  provide  the  management  as  part  of  the  maintenance  contract.  This  practice  is  slow  to  change 
because  the  manufacturers  do  not  want  to,  as  they  can  capitalize  on  services  revenue;  and  customers  do  not 
know  how  to,  because  there  is  no  off-the-shelf  product  on  the  market  as  an  alternative  to  the  manufacturers’ 
offerings. 

The  obvious  reason  for  the  lack  of  3rd-party  monitoring  tools  for  medical  equipment  is  due  to  the  proprietary 
hardware  and  software  interfaces.  Even  worse  is  the  incompatibility  of  the  proprietary  protocols  between 
equipment  from  the  same  manufacturer.  As  long  as  the  manufacturers  are  not  willing  to  release  their 
specifications,  and  they  continue  to  use  proprietary  protocols  in  their  hardware  and  software  interfaces,  there 
will  be  no  standardization. 

The  not-so-obvious  reason  is  the  fact  that  some  complex  medical  equipment,  such  as  the  CT  scanner,  may 
have  two  management  points:  1)  the  “computer”  part  that  runs  a  specialized  OS,  such  as  the  Microsoft  XP 
Embedded,  along  with  custom  software  used  by  the  clinician  to  operate  the  equipment.  For  simplicity,  we  call 
this  the  “in-band”  management  of  the  medical  equipment.  And  2)  the  “mechanical”  part  that  is  controlled  by 
circuit  boards  and  is  accessible  by  technician  via  the  serial  console  and  running  proprietary  diagnostic 
software.  Again  for  simplicity,  we  call  this  the  “out-of-band”  management  of  the  medical  equipment. 

5.2.1  The  In-band  Management 

The  in-band  management  point  is  actual  a  standard  computer  housed  in  a  custom  case  and  thus  can  easily  be 
managed  by  existing  IT-centric  monitoring  tools.  This  is  because  the  OS  is  Windows  based  and  the  services 
running  on  this  computer  is  network  based.  As  such,  as  long  as  the  computer  is  fully  operational,  any 
monitoring  tools  can  be  used  to  check  its  CPU,  memory,  disks,  processes,  and  custom  applications. 

5.2.2  The  Out-of-band  Management 

The  out-of-band  management  point  is  only  accessible  via  the  serial  console  on  the  medical  equipment. 
Furthermore,  the  statistics  and  the  operational  state  of  the  equipment,  in  most  cases,  can  only  be  obtained 
using  the  proprietary  software  that  runs  on  the  technician’s  computer.  The  technicians  generally  use  the 
software  to  configure  and  calibrate  the  mechanical  parts  of  the  medical  equipment. 

With  two  separate  management  points,  the  statistics  gathered  can  be  deceiving  if  only  the  in-band  component 
is  being  monitored  via  off-the-shelf  IT  monitoring  tools.  This  is  because  while  the  “computer”  portion  may 
be  healthy  with  no  apparent  sign  of  trouble,  the  mechanical  part  may  be  severely  mis-calibrated,  requires 
oiling,  or  is  experiencing  high  temperature  that  may  be  damaging  to  the  equipment,  if  left  neglected,  or  may 
produce  erroneous  patient  test  results. 
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5.3  Proactive  Monitoring  System  Development 

The  objective  of  the  research  is  to  evaluate  off-the-shelf  technology  that  can  be  leveraged  in  developing  a  new 
monitoring  platform  that  can  be  programmed  to  emulate  medical  equipment  technicians  that  systematically, 
automatically,  and  proactively  accesses  the  medical  equipment  to  monitor  their  operating  state.  This  system 
can  be  scheduled  to  perform  checks  when  the  medical  equipment  is  not  in  use  or  when  the  technician  is  not 
available  on  site  to  perform  maintenance  over  an  extended  period  of  time. 

In  order  to  solve  a  problem  with  infinite  diversity,  due  to  the  proprietary  hardware  and  software  interfaces  on 
medical  equipment,  and  with  no  access  to  any  specifications,  protocols,  API,  or  SDK  from  the  equipment 
manufacturers,  the  strategy  is  to  identify  the  commonality  and  predictability  that  exists  in  medical  technology. 
There  is  one:  the  human  element  is  a  consistent  and  reliable  criterion  that  medical  equipment  manufacturers 
must  take  into  consideration  when  they  engineered  their  products.  Namely,  it  is  the  software  user  interface, 
designed  for  human  interaction  with  the  medical  equipment,  as  the  basis  for  our  project  design  focus. 

The  process  of  simulating  a  trained  technician,  that  is  connected  to  the  serial  console  of  the  medical 
equipment  and  is  navigating  the  proprietary  software  to  perform  specific  maintenance  tasks,  can  be  broken 
down  into  three  key  steps:  navigation,  observation,  and  action. 

Navigation  represents  the  sequence  of  key  presses  and  mouse  clicks  over  an  application  UI  visible  on  the 
technician’s  computer  screen.  Each  of  his  activities  is  an  input  and  the  screen  change  (rendering  of  a  new 
page),  object  change  (a  button  is  depressed  after  his  click),  character  painting  (echo  of  his  typed  text),  etc.  are 
outputs. 

Observation  represents  his  viewing  of  the  presentation  on  the  screen.  If  the  software  is  not  waiting  on  a 
backend  process  to  complete  (ie:  no  hour-glass  on  the  screen),  then  this  is  “idle”  time.  Otherwise,  it  is  “wait” 
time. 

Action  represents  his  activities  after  the  idle  time  starts.  If  the  activity  is  key  presses  or  mouse  click  in  the 
application,  then  he  is  back  to  the  Navigation  step.  If  his  activity  is  outside  the  application,  he  may  be 
performing  a  remediation  for  a  problem  he  saw  during  the  Observation  step.  While  auto-remediation 
capability  can  be  added  to  this  project  at  a  later  time,  it  is  outside  the  current  scope  of  the  “monitoring” 
objective. 

We  are  currently  developing  the  monitoring  capability  based  on  the  navigation  and  observation  steps.  This 
means  the  system  can  act  as  the  technician  in  traversing  the  UI  and  reviewing  the  results. 

5.3.1  Developing  Navigation  Capability  with  Macro-based  Automation 

One  existing  technology  that  is  commonly  used  by  IT  administrator  is  the  macro-based  automation  tools.  This 
type  of  tools  is  designed  to  record  and  replay  a  user’s  activity,  within  an  application  running  in  a  graphical 
desktop  environment,  such  as  Windows,  to  automate  repetitive  tasks  without  programming.  Out-of-the-box, 
this  type  of  tool  is  not  efficient  because  a  recorded  segment  can’t  be  easily  modified.  For  example,  if  there  are 
10  steps  to  perform  task  A  and  there  are  1 1  steps  to  perform  task  B  (the  first  10  steps  are  identical  as  task  A), 
the  user  has  to  create  a  new  macro  from  scratch,  recording  the  first  10  steps  before  doing  the  11th.  However,  it 
is  possible  to  enhance  an  existing  macro  tool  to  overcome  this  limitation  by  manipulating  the  recorded 
segment  in  a  macro. 
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With  customizations  and  enhancements  done  to  an  off-the-shelf  macro  tool,  we  will  be  able  to  train  a 
computer  (once),  by  recording  the  activities  of  a  trained  technician  performing  a  set  of  maintenance  checks; 
we  can  then  subsequently  edit  and  tag  certain  segments  to  create  macro  sets  for  navigating  to  various  areas 
within  the  proprietary  diagnostic  software.  When  each  macro  is  replayed  at  scheduled  intervals,  the  result  is 
simulating  the  navigation  of  the  technician  for  performing  a  specific  maintenance  check  on  the  medical 
equipment.  This  is  the  “navigation”  capability  of  the  monitoring  system. 

5.3.2  Developing  Observation  Capability  with  Computer  Vision 

One  existing  methodology,  known  as  screen  scraping,  is  a  technique  in  which  a  program  is  designed  to  extract 
data  from  the  “display”  output  of  a  second  program,  whereby  the  second  program  does  not  have  to  be  running 
on  the  same  computer.  The  significance  of  this  type  of  data  extraction,  which  is  different  from  parsing  data 
between  applications,  is  that  screen  scraping  collects  data  that  is  in  the  final  presentation  state  of  the  data 
designed  for  human  viewing.  This  method  is  popular  for  application  developers  to  interface  with  a  legacy  or 
3ld-party  software  without  having  access  to  the  protocols,  data  structure,  or  API. 

Utilizing  screen  scraping  technique,  we  can  implement  a  method  of  “observation”  for  our  monitoring  system. 
In  the  previous  navigation  step,  a  macro  represents  the  sequence  of  activities  performed  by  the  trained 
technician  to  go  from  the  start  of  the  UI  to  a  destination  where  he  then  “observes”  the  data  output  on  the 
screen.  The  screen  that  the  technician  is  viewing  has  been  captured  as  an  image.  The  screen  scraping  step  is 
essentially  an  image  manipulation  process,  based  on  neural  networks  programming,  designed  to  quickly 
obscuring  the  majority  of  the  graphical  area  and  identify  all  the  “useful”  areas  based  on  various  algorithms. 
Useful  areas  are  defined  as  tab  labels,  button  states,  fixed  fields  (grayed  out),  editable  text  fields  (fields  that 
are  changeable),  and  etc.  Mimicking  a  human,  this  process  can  accurately  identify  all  “useful”  areas  because 
there  is  a  finite  set  of  object  that  can  appear  in  an  application  UI. 

Once  the  screen  image  has  been  fully  “marked”  or  “dissected”,  the  screen  scraping  process  continues  to  the 
next  stage:  data  recognition.  This  process  is  known  as  Optical  Character  Recognition  (OCR).  OCR 
technology  has  existed  for  many  years  and  one  common  use  is  extracting  contact  information  from  business 
cards.  OCR  is  not  effective  when  it  is  used  on  handwriting,  printed  characters  that  use  non-standard  fonts,  or 
large  image  area  (such  as  translating  a  faxed  document  into  editable  content).  In  the  case  of  processing 
application  UI  screen  images,  focusing  only  on  the  marked  “text”  areas,  the  speed  can  be  extremely  fast  and 
can  achieve  100%  accuracy  because  of  the  small  area  to  scan  and  the  fonts  in  application  UI  are  very 
common,  respectively. 

When  the  OCR  process  completes,  the  extracted  data  is  mapped  to  a  predefined  structure  in  the  database 
based  on  the  macro  used;  and  subsequently,  an  XML  file  can  be  generated  for  exporting  the  data  to  be  further 
processed  by  an  external  program.  For  the  monitoring  process,  the  XML  file  can  be  a  formatted  and  forward 
to  a  syslog  server,  a  snmp  collector,  or  to  an  enterprise  monitoring  system. 


6.0  CONCLUSION 

Since  last  2008,  the  research  and  development  efforts  for  medical  equipment  RDA  and  CBM+  enablement 
have  created  the  framework  for  the  telemaintenance  and  condition-based  maintenance  transformations.  This 
framework  can  be  expanded  overtime  to  include  and  adopt  other  IT  best  practices  in  operations  processes, 
security  measures,  and  resource  manageability. 
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The  Teleconsole  provides  a  centralized  and  standardized  RDA  method  in  a  single  platform  to  access  virtually 
any  device  regardless  of  the  complexity,  brand,  make  or  model.  The  RDA  function  paves  the  way  for  the 
Condition-based  maintenance,  which  allows  for  proactive  maintenance  and  management  of  the  equipment  - 
leading  to  true  telemaintenance.  The  new  telemaintenance  solution  will  improve  the  availability  and 
resiliency  of  the  medical  equipment,  reduce  costs  associated  with  unscheduled  repairs,  and  validate  equipment 
performance  measures,  which  in  turn  provides  the  physicians  with  the  ability  to  deliver  quality  health  care  to 
their  patients. 
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